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ABSTRACT 

This volume 



presents conclusions and implications derived from five cities 
experience with performance contracting In education: Norfolk, Virginia; Texarkana, 
Arkansas (with L i berty-Ey 1 au , Texas); Gary, Indiana, Gilroy, Ca 1 i forn i a ; and 
Grand Rapids, Michigan. The five case studies cover eight programs in 15 schools 
While each study is treated in a self-contained report, it has also been part of 
a coordinated investigation of more than 20 programs conducted from 1969_to 1971* 
This volume begins with a brief description of Rand's field study. lmpl|cations 
are then discussed under seven headings: instructional processes, cognitive 

growth, resource requirements, evaluation, program management, returns to 
contractors, and the major advantages and disadvantages of performance 
contracting. Conclusions include: (l) Individualized I nstruct s on was the goal of 

all programs; (2) The performance contracts, In most cases, produced respectable 
gams on standardized achievement tests; (3) Costs of programs vary ividely; (4) 
Performance contracting fostered a healthy emphasis on the student and his 
learning as a measure of program success; (S) Performance contracting Is proving 
to be a useful research and development tool; (6) This contracting has not 
generated large profits so far; (7) Performance contracting facilitates ? r.e 
““introduction of radical change in education; (8) Some programs have been so 
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PREFACE 



This Report is a product of Rand^s study of performance contracting in educa- 
tion, The study is sponsored by the Assistant Secretary for PJanning and Evaluation, 
U.S, Department of Health, Education and Welfare, under Contract hio HEW=OS= 
70-156. 

Case Studies in Educational Performance Contracting comprises si% volumes. 
Each is a self-contained study; together they provide a multifaceted view of perform- 
ance contracting. The six volumes are: 

1. R'900/1»HEW, Conclusions and ImpUcatiotiSf by P. Carpenter and 
G. K Hall 

2. R-900/2-HEW, Norfotkf Virginiaf by P. Carpenter 

3. R-900/3-HEW, Texarkana, Arkansas and Liberty-Eylau, Texas, by 
P. Carpenter, A. W. Chalfant, and G. R. Hall 

4. R-900/4-HEW, Gary, Indiana, by G. R. Hall and M. L. Rapp 

5. R-900/5-HEW, Gilmy California, by M. L. Rapp and G, R. Hall 

6. F 300/6“HEWj Grand Rapids, Michigan, by G. C. Sumner 

This study is the second of three Rand Reports on the subject. The first Report 
was J. P. Stucker and G, R. Hall, The Performance Contracting Concept in Education, 
The Rand Corporation, R-699/1-HEW, May 1971. The third Report will be a per- 
formance contracting guide intended for use by educational officials. 
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This volume presents conclusions and implications derived from five cities ex- 
perience with performance contracting in education: Norfolk, Virginia; Texarkana, 
ArkanspiS (with Liberty-Eylau, Texas); Garyj Indiana; Gilroy, California; and Grrnd 
Rapids, Michigan, The five ease studies cover eight programs in 15 schools. While 
each study is treated in a self-contained Report, it has also been part of our coor- 
dina ted investigation of more than 20 programs conducted from 1969 to 1971. 

This volume begins with a brief description of Rand’s field study. Implications 
are then discussed under seven headings: instructional processes, cognitive gi owth, 
resource requirements, evaluation, program management, returns to contractors, 
and the major advantages and disadvantages of performance contracting. 



INSTEUCTIONAL PROCESSES 

We have arrived at five conclusions concerning the influence of performance 
contracting on instructional processes: 

« Individualized instruction was the goal of all programs. 

■ Because the programs focused on the disadvantaged, there was an emphasis 
on basic skills and considerable use of programmed materials. There were coneomh 
tant problems in providing for abler students. Programs wiU continue to be skilh 
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oriented, because of the stress on remedial training and thi? lack of consensus about 
the objectives of instruction outside of the simple cognitive skill areas. 

# There was no evidence of dehumanization; there was some evidence of the 
reverse. 

« Performance contracting programs have been and wi.4 probably continue to 
be developmental efforts rather than applications of off-the-rihelf systems. 

• Performance contracting was an educational change agent in the programs 
ws observed. 
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COGNITIVE GROWTH 

We have drawn four conclusions about cognitive growth (loughly speaking, 
growth in intellectual capabilities): 

• The performance contracts did not produce dramatic gains on standardized 
achievement tests, but in most instaaces gains were respectable, 

• The performance contracting movement has focused attention on the prob- 
lems associated with gain scores on standardized achievement testci, 

• Much more work needs to be done on criterion-referenced te^ ts before their 
results can be interpreted meaningfully. 

• Performance contracting has demonstrated that decisionmakers responsible 
for passing judgment on program effectiveness must become more involved in choos- 
ing instruments that will measure that effectiveness. 



eesouece requirements 

Costs of the lirograms were analyzed by means of a model that yields compara- 
ble replication coscs-- that is, costs derived from a single set of prices for resource 
unite- Three conclusions v/ere drawn: 

• Comparable replication costs of performance contracting programs vary 
widely. In the Rand sample, per-student, per-subject cost varied as much as 80 
percent. 

• Performance contracting programs cost more than conventional instruction. 
This is to be expected, since their piirpose is to make up for the educational disadvan- 



vi 



j 

1 

ij 



I 






tages of target students. 

# Performance contracting programs cost about the same as (or less than) 
typical remedial programs funded under Title I of the Elementary and Secondary 
Education Act. This is because the performance contracting programs substitute 
aides, materials, and equipment for highly trained and highly paid special teachers. 

* The important consideration, however, is not cost^per-student but cost-efiec- 
tiveness. Unless future performance contracting programs achieve higher cognitive 
gains than past programs have, they will have to be justified on i.he basis of ancillaiy 
Ijenefits such as curriculum development potentials. 




EVALUATION 



Four major points concern the evaluation of the prograrns. 

« Performance contracting fostered a healthy emphasis on the student and his 

learning as a measure of program success. 

• The requirement for maintaining the integrity of the validation of scores on 
achievement tests sometimes made it difficult to uss evaluation data for program 
impiovement. 

• Evaluation designs were often haphazard or nonexistent. 

I Data needed for thorough evaluations were usually inaccessible or unavaila- 
ble. 



PROGRAM MANAGEMENT 

The section on program management considers six points: 

. Performance contracting is proving to be a useful research and development 
tool. People who are not a permanent part of the school system seem to be freer to 
implement radical changes in the classroom than are regular school personneL 

• A respected and influential "sponsor” within the school district is a great 
help in overcoming inherent frictions and impediments to change. To assure that 
changes are permanent and that they expand beyond isolated programs will require 
continued high levels of sustained effort by the Local Educational Agency (LEA). 

• Flexibility is essential in program organization and management, since con- 
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siderable program development will take plaee. Multiyear programB have advan= 
tages over single=year programs fi r this reason. 

• Performance contracting piograrns impose special tasks of management and 
coordination not only on contractors but on the schools’ administrative personnel. 
The complexity of some programs has exacerbated these pi oblems. School adminiS' 
trators must be prepared to face legal and labor disputes. Most of these can be 
resolved, but there are two potential areas for serious conflict. One is the require^ 
ment for public control of all school programs. The other is teacher opposition to 
merit pay. 

• It seems essential that local teachers be involved in program design and 
administration. 

• Little effort was made in most programs to inforin parente about the pro= 
grams or to involve them. Many parents were confused by or hostile to some aspects 
of some programs. 



RETURNS TO CONTRACTORS 

With regard to the returns to contractors, four eonciusions were reached: 

• Performance contracting does not seem to have generated large profits so far. 

• Performance contracting has generated some follow-on programs, only some 
of which tie fees to student achievement. 

• Established contractors tend to prefer other arrangements to performance 
contracting, such as consultantships. Performance contractors will seek to convert 
performance contracts to other types of programs. 

• Performance contracts have enabled a number of firms to break into new 
markets and to receive publicity for their goods and services. 



MAJOR ADVANTAGES AND DISADVANTAGES 

From the foregoing outline, we have identified three major advantages and 
three major disadvantages of educational performance contracting that can be in- 
ferred from the results of our field work. The following are the major advantages 
as we see them: 
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• Performance contracting facilitates the introduction of radical change in 
education* 

« It places increased emphasis on accountability for student learning on the 
part of school administrators, contractors, and teachers. 

■ It has brought new Learning System Contractors (LSCs) into the educational 

field* 

Three disadvantages also seem evident: 

- Some performance contracting programs have been so complex that manage- 
ment has been severely hampered and costs have been unnecessarily high. 

• Performance contracting programs will probably continue to be narrowly 
focused because of difficulties of defining objectives in subject areas other than those 
involving simple skills or, in some cases, difficulties in measuring the attainment of 
objectives. 

• Performance contracting has exacerbated old problems to the point where 
they almost seem to be new ones. The most severe have been legal questions, issues 
of teacher status, difficulties in supplying the needed management skills, and espe- 
cially, problems of test selection and administration* 
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I. INTRODUCTION 



THE RAND/HEW STUDY 

The RANP/HEW study objective is to provide guidelines for decisionmakers in 
Local Educational Agencies (LEAs) helping them determine whether to enter into 
performance contracts and to use them eftectively if they do. A fundamental tenet 
of the study has been that these guidelines must be developed from dstailrd analysis 
of actual programs; therefore, much of the research effort has been devoted to 
studying eight performance contracts involving seven learning sysi^m contractors 
(LSCi), 1C schools, and six school districts in five cities: Norfolk, Virginia; Texar- 
kana, Arkansas (with Liberty-Eylau, Texas); Gary, Indiana; Gilroy, California; and 
Grand Rapids, Michigan. The purpose of this volume is to evaluate or compare 
the specific contracting programs, as such, but to draw implications from these 
programs that may be of general use. 

Initially, there was some thought that a single cross-district study format might 
be useful. As the planning in the various districts progressed, however, it rapidly 
became apparent that the projects would difTer so much that this approach would 
be inappropriate. It was also apparent that some of the most instructive experiences 
— such as the dispute with the teachers* union in Gary — ^were specific to one or a few 
programs. Therefore, Rand designed a plan for a field study in depth of a small set 




of programs, supplemented a less intensive look at a larger group. ^ This volume 
summarizes Raiid^s field monitoring work with primary emphasis on the eight 
programs involved in the case studies, 



THE FIVE CASE STUDIES 

The sample was chosen to provide a diverse group of school districts and pro^ 
grams. Four geographic regions are represented: Southeast, North Central, South 
Central, and Far West^as shown in Fig, 1. The cities also vary widely in size of the 
total population and of the enrollment in public schools, as illustrated in Table 1, 
Both urban and rural areas are represented, as are black and Spanish-surname 
minority groups, as well as whites. 

Table 2 presents some general features of the eight programs. Note that, unlike 
the usual demonstration program in education, each of the programs except for the 
one in Gilroy involved a relatively large number of students; each, except for the one 
in Gary, was limited to the skill areas of reading or reading and math, and only one 
(Gary) involved an entire school. One LSC was a subsidiary of a major textbook 
publisher, one was primarily an educational hardware firm, and the other five were 
curriculum and educational services firms. All were active in performance contract^ 
ing elsewhere. Half the contracts were let by competitive bid and half by sole-source 
negotiation. Some of the programs entailed elaborate contractual arrangements, 
including management support contractors and contracted evaluations and audits. 
In others, all planning, management, and evaluation was done by school district 
personneL 

The Alpha Systems contract in Grand Rapids was part of the Office of Economic 
Opportunity's experiment in p3rformance contracting that involved 20 programs 
throughout the country.^ The much-publicized program in Texarkana during the 

* The performanca contracting of LEAs and educational authoriti^ (SEAs) have 

inciuded, in addition to the above-mentioned programs: Alachua County, Fla,; Colorado programs (three 
in Denver area sponsored by SEA); Compton, Calif.; Flint, Mich.; Greenville, S.C,; Mesa, Ari 2 .; Muskegon, 
Mich.; Philadelphia, Pa.; Portland, Ore.; San IMego, Calif.; San Francisco, Calif; Vir^nia programs 
(including activities of SEA); Yuba County, Calif. 

* For further infdrmahon, §« T. K. Olennan, ' 'OEO Experiments in Education," Compact^ Vol. 5, No. 
1, February 1971, pp. 3-5, or J. O. Wilson; "Statement Submitted for the Kecord on Performance Contract^ 
ing Before the Houie Committee on EducaUon and Labor,'* Office of Elconomic Opportunity, Washin^sn, 
D.C., April 30, 1971 (mimeograph). 
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Fig. 1— The five case studies 



1969-70 school year was included, as well as the 1970-71 program, the only program 
that included a turnkey component (an effort to transfer the contracted program to 
the regular school curriculum). The program in Gary was also one of the case 
studies; it was also highly publicized because a contractor operated an entire school, 
giving rise to difficulties with the teachers union and the state department of educa- 
tion. 

Some of the significant features of the contracts themselves are displayed in 
Table 3. The performance guaranteed by the contractor was usually on the order of 
"one grade-level” gain in about a year of instruction.® LRA in Norfolk had the most 

® BRL in Gaiy speciiied performance “at grade level” for itudents in the program for 3 years. For- 
thoia in the progrmn tbr legs time, the goal is one grade-level gain per year. 




Table 1 

characteristics of case-study cities 



city 


1970 

Population 


1970 

Public 

School 

Enrollment 


Minority 


percent 

Minority 


Econo^ c Base 


Gary 


182,000 


47,000 


Black® 


60 


Manif ae tur ing 


Gilroy 


11,250 


5,000 


Spanish surnama 


3/i . 


Agric ilture 


Grand Rapids 


200,000 


41,000 


Black 


20 


Manufacturing 


Norfolk 


300,000 


55,000 


Black 


30 


U , S . Navy 


ksna 


6Q,0GG 


14,000 


Black 


27 


Manuf ae tur Ing 



^Also has a sizable Spanish=surnama coramunltys 



ambitious objective: student gains of at least 1.7 grade levels by the end of the 
program. Criteria for student selection also varied from a very loose "all assigned 
by the LEA” to the complex specifications of the Dorsett and EDL programs in 
Texarkana. Evaluation designs ranged from the casual, in-house plans for the WLC 
programs in Grand Rapids and Gih’Oy to the elaborate evaluation plans for the EDL 
program in Texarkana involving independent evaluation contractors. Provisions for 
contract changes were specified in only half the programs, and the provisions in 
Texarkana/ Dorsett were minimal. Provisions for turnkey programs received even 
less attention in all programs except Texarkana. 



GENERAL FEATURES OF THE FIELD STUDIES 

The Rand research team drew up a plan for the field studies at an early stage 
of the work. It included schedules of visits to the districts selected for research in 
depth, detaUed lists of data to be collected, guides for classroom observation, and 
categories of people to be interviewed. The plan was kept flexible because it was 
antidpated that program-changes during the year would force changes in the study. 
The field study was a team effort involving people from several professional 
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disciplines: education, psychology, economics, business administration, and system 
anaiysis. In essence, however, a single team member was responsible for the field 
study in a given city. This fostered a continuity of interaction between the Rand 
team member with primary responsibility and the school district and contractor 
personnel. It also assured a continuity of study effort. 

The researchers assigned to each district used the master plan to develop a 
localized plan that reflected special features of the LEA and LSC, and the educa= 
tional or political issues that appeared to be of the widest general interest. Natu- 
rally, the studies also reflect the professional backgrounds and interests of the 
researchers. In short, each case study, reported in a separate volume of this series, 
has unique features. While all the case studies reflect a common or basic plan, no 
attempt was made to homogenize them nor to maneuver the team members into a 
consensui concerning conclusions and implications. 



PLAN OF THIS VOLUME 

This volume presents implications that are common to the individual studies. 
The objective is not to pass judgment on each of the programs as such, but to draw 
from them useful information on perfonnance contracting as a technique for organ- 
izing instructional programs. These implications are discussed under the following 
headings: instructional processes, measuring cognitive achievement, evaluation, 
resource requirements, program management, and returns to the contractors. The 
final section sums up the team*s views of the m^or advantages and disadvantages 
of performanc e contracting. 



II. INSTRUCTIONAL PROCESSES 



Two intertwined hopes have supplied much of the impetus for performance 
contracting. One has been that it might serve as an instrument for change, stiniulat= 
ing innovations in education, v, hich has lagged behind most other fields in the rate 
of technological change. Central to this hope is the concept of turnkeying eost= 
j efiTective new educational processes. Performance coiitractors, it has been theorized, 

I would be able to demonstrate their systems under field conditions/ A school system 

' with an elective performance contract could later enter a turnkey phase, in which 

I it took over the new instructional S3rst0m for its own use. 

i The second hope has been that performance contracting would help solve 

America’s compensatory education problem. School ofiUcials, particularly since the 
Cclaman Report/ have been under great pressure to raise academic achievement, 
especially among disadvantaged socioeconomic groups whose membei^ often leave 
schools with severe educational handicaps. Public attention has been focused on 
cognitive achievement as measured by standardized test scor^. Conventional 
remedial education having had a disappointingly low impact on these scores, per- 

On the turnkey concept see Ekluuution Turnkey Systems, Inc., Pgrformanjce Contracting in JEefuca- 
twn. Research Pr^, Champaign, Illinois, 1970; L. M. Lessinger, Evsiy Kid A Winner: Accountability in 
Education, Simon and Schuster, New York. 1970. 

* J. E. Coleman, et al.. Equality of Educational Op^rtunily, U.S. Office of Education, Washington, 
D.C., 1966. 
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formancQ contracting has been looked to as the possible answer. Its potential 
remedial powers could work either directly or indi. ectly. Directly, contractors claim 
to possess proven instructional systems that, if properly used, are capable of produc- 
ing drarnatic increases in achievement scores. Indeed, the contractors have been so 
sure of results that they have been willing to ^'guarantee'* substantial improvements 
in standardized test scores^ Harassed public school administrators find contractors 
offers to ''do something” about low-achieving students most appealing. 

Indirectb^j it is argued, performance contracting facilitates the introduction of 
materials and instructional methods better suited for those students who now score 
poorly on achievement tests. Perhaps, the reasoning goes, low achievement on stand- 
ardized tests is the fault of using conventional textbooks and teaching methods 
designed for middle-class children but bordering on the irrelevant for lower socioeco= 
nomic groups. If so, it follows that new equipment, materials, and incentives are 
needed for teachers or students. The problem is how to put effective new systems into 
use. 

This section examines the impact of performance contracting on instructional 
processes. Since the subject is explored in detail in several of the case studies, it will 
be merely summarized here. The next section considers the impact of performance 
contracting on cognitive achievement. 

Despite the problems of implementing change in the public schools and the 
frequently virulent opposition to performance contracting, it has succeeded in bring- 
ing about change. For several reasons, however, the concept haii lost almost all of 
its apparent simplicity in application. First, the programs in our sample of eight 
were development efforts, rather than mere transplants of fully developed learning 
systems.” In every case, substantial revisions were needed as the program pro- 
gressed. Curricula had to be changed because the student populations differed from 
those the instructional programs had been designed for; lines of authority and 
responsibility had to be adjusted to achieve harmonious relationshii^ among the 
LEA, teachers, and parents or other involved parties; schedules had to be i-eviBed 
because students needed more (or less) time with the program than had originally 
been estimated; students did not respond as expected to motivational techniques; 
teachers who could not cope with the changed instructional process had to be re- 
placed. It should not be surprising that an instructional program would need to be 
revised, sometimes radically, when put to use in a new school system. LEAs differ 
widely not only in the nseds ot their students but in the ability ol the school system 
to support any particular program. A program must be adaptable to a wide range 
of many variables if It is to be of general use. Currently, at least, it would be naive 
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to believe that an LSC could install a useful, completely preps, ckaged progran?: in an 
unfamiliar school district. 

Because the range of capability of the students in most programs was fairly 
wide, and because the contractors wished to make full use cf each student*s time, 
the programs were all built around the concept of individual] zed instruction. More- 
over, because the contractors wished to demonstrate efficient instruction, materials 
or equipment tliat fostered self-instruction were used. The basic approach, then, was 
to determine what each student did and did not know about the subject of instruction 
(diagnosis) and then to assign materials from the instructional program to teach hi a* 
what he did not know (preseription). Programs suffered, of course, when the diagnos- 
tic /prescriptive materials were inappropriate for the student population or, worse 
yet, when they were not all available from the outset of the program. In programs 
with a broad range of student capabilities, some contractors were hard put to provide 
enough variety to meet the needs of students at the far ends of the distribution. 
Nevertheless, almost ail programs had achieved a substantial degree of individuali- 
zation by the year\s end. 

To facilitate self^inst ruction, programmed materials were extensively used. 
Some students found these dull, but many found them exciting and took a greater 
interest in school. These materials seemed tc appeal more to students at the lower 
levels of academic sophistication, however — that is, at the lower ages and grades or 
the lower achievement levels. This may have bee n as much the fault of the way in 
which the materials were programmed m it was of the fact that they were pro= 
grammed. After all, programmed materials that challenge and excite the bright 
student have been writteri. 

Some, but not most, of the programs featured teaching machines. Of the eight 
programs studied, two (Dorsett and EDL in Texarkana) were very machine-centered. 
CMES (Grand Rapids) relied heavily on m.achines, but also used supplementary 
materials. WLC (Gilroy and Grand Rapids) and LRA (Norfolk) used tape recorders 
along with a wide variety of paper-and-pencil materials. BEL (Gary) had a paper- 
and-pencil 83^teni, although there wais a modest reading laboratory using some 
equipment during the morning at Banneker school. Alpha (Grand Rapids) also took 
a nonmachine approach. (See Table 2 for the full forms of abbreviated names.) 

Because self-instruction was the heart of the instructional program, several 
contracts featured special incentives for students. Extrinsic motivators for accom- 
plishing assigned learning tasks were used in the Alpha, WLC, and Dorsett pro- 
grams. CMES allowed students time in reinforcement centers and spent a substan- 
tial fraction of instructional time working directly on students* self images and 
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goals. LRA used special rewards only for student-initiated outside reading, arguing 
that academic success provides its own rewards and incentives. ERL and EDL made 
no use of special incenti^/es. 

All programs aimed at teaching specific skillSi primarily reading and math- 
ematics. 7L'his was a natural ccnsequence of using scores on achievement tests in 
reading a.nd math as a measure of the quality of education, because of the relative 
ease with which desirable skilis in these areas may be identified and assessed. LSCs 
believed their systems were etfective enough that they could demoiistrate marked 
improvement in student learning in the skills areas. Even though the Gary program 
involved subjects other than reading and mathematics, payments to BRL were based 
on student scores in reading and mathematics, and the curriculum emphasized these 
subjects. The emphasis on testable skills bias been criticised by soma. This criticism 
is probably justified if we aie speaking of students who already have some mastery 
of the basic skills; it seems less reasonable for students who do not. Some students 
in some of the programs could not read at all, for example. 

There was concern that performance contracting would be "dehurnanizing”™ 
that is, it would force the student into a mechanized environment in which he would 
produce to satisfy the profit motive of the contractor. We could find no evidence that 
it was, and did find some significant evidence of the reverse. First, the individualized 
approach of most systems required tliat the student accept greater personal respon- 
sibility for his activities. It has not always been easy to foster the necessary selfi 
reliance, but when students do take the initiative for their own learning they seem 
to develop more maturity and respect for themselves and others. The contrast in this 
respect with students in regular classrooms was apparent in several programs. 

Secondlyi individualization and programmed instruction led to classroom reor- 
ganization, with less of the “teacher at the head of the class” syndrome. Teachers 
and pupils tended to interact more informally, in e more warmly personal way. This 
easy informality was even more readily attained in the many programs that liad 
aides who handled routine matters and in some eases instruction. Contractors dejib- 
erately fostered the informal approach and strove to stimulate enthusiasm because 
they were convinced that unhappy students do not learn. BRL in Gary tried to 
promote what it termed a “more humane” atmosphere because it thought it would 
lead to better academic performance, but was hampered by parental dislike of 
“permissiveness. ' ’ 

Teachers in Gilroy told us they disapproved of turning the teacher into a diag- 
nostician and curriculum manager and decreasing her role in the materials-teacher- 
student chain. We found the WLC center personnel providing some tutoring on a 
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one-to-one or smalUgroup basis when they felt that certain students would benefit 
from a more personal approach. The organizational format of the program permit* 
ted such an approach. 

To repeat, then, we discovered no real evidence of dehumanization in the pro- 
grams we studied, and have seen some evidence of the reverse.® In sum: 

• Individualized instruction was the goal of all programs. 

• Because the programs focused on the disadvantaged, there was an emphasis 
on basic skills and considerable use of programmed materials. There were concomi- 
tant problems la providing for abler students. Programs will continue to be skill- 
oriented, because of the stress on remedial training and the lack of consensus about 
the objectives of instruction outside of the simple cognitive skill areas. 

• There was no evidence of dehumanization; there was some evidence of the 
reverse. 

e Performance contracting programs have been and will probably continue to 
be developmental efforts rather than applications of off-the*shelf systems, 

• Performance contracting was an educational change agent in the i)rogramB 
we observed. 

® In the Texarkana program, many students apparently felt they had been relegated to "dummy 
classe.s‘*— a stigma often attached to remedial programs. Unfortunate as this stigma is (and certainly off 
the mark in the case of Texarkana), it can scarcely be equated with dthumanization. 
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III. COGNITIVE ACHIEVEMENT IN PERFORMANCE 
CONTRACTING PROGRAMS 



To date, performance contracts have been covenants between LEAs and LSCs 
whereby the latter are compensated for their services largely on the basis of stu- 
dents’ cognitive growth as measured by standardized tes^. The measured gains for 
the programs in the sample are summarized in Table 4, 

The letters "NR’’ in Table 4 stand for "not released.” In the Texarkana-Dorsett 
case, the two parties are negotiating a settlement and litigation is likely. Final 
.results will not be released until the matter is settled. Even then, according to the 
evaluator and auditor, the validity of the statistical data lias been nullified by 
"teaching to the test.” In the otlier programs the results will be available when they 
have been checked and rechecked and all financial claims involving them have been 
resolved. Educators are used to prompt release of achievement=test features. Cau- 
tion and delay in release of test scoi-es in performance contracting programs is an 
inherent feature because of their legal significance. 

The numbers in Table 4 are cognitive gains expressed in achieveraent-yeai^ as 
measured by standardized tests. A figure of 1.0 would represent one year’s growth 
as measured by the difference between a pre- and post-test. While 1,0 is the average 
for all students, populations "uch as those involved in these programs more typically 
have yearly gains of around 0-6 to 0.7. 

Before discussing the implications of these gain scores, we should note that only 
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the data for Gilroy and for the fifth grade in Norfolk represent gains by the majority 
of the students in the program. The CMES and Norfolk junior high data represent 
gains for only about a third jf the students, because of difficulties in obtaining both 
pre- and post-tests for students and high absence rates in these grades. The gains 
by other students in the CMES program o e not known to us; the Norfolk formula 
ibr payment to LRA (computed from averages of all pre- and post-test scores) at- 
tributed a mean gain of 0.9 to the students at the junior high level The WLC 
program in Grand Rapids was expanded at midyear; therefore, the gain scores 
reported here represent only a third of the students (probably the less able students) 
ultimately in that program. 

The gains listed in Table 4 are generally respectable but, except for Gary and 
CMES in Grand Rapids, they were below contractor expectations. There were expec- 
tations, for example, that students would gain two grade levels in a single year. In 
some cases, such as the LRA program for the fifth grade in Norfolk and the EDL 
program in Texarkana, the gain scores are poorer than those the schools had been 
registering prior to performance contracting. In other cases, such as the WLC pro- 
gram in Grand Rapids and the LRA program at the junior high level in Norfolk, the 
scores were about the same. In other programs the picture has been more complex. 
In Gilroy, for example, the WLC canter students in grades 3 and 4 did better than 
other Title I students in reading; in grade 3 mathematics, Title I students did better 
than the WLC program students, but in grade 4 the WLC students did better. 

In view of this picture, the gains in Gary are unusually good. We do not think 
this is attributable to the fact that an entire school was used, as the LRA program 
encompassed almost the entire fourth through sixth grades in Norfolk. We also do 
not think that the students at Banneker were significantly mure able than those iu 
other programs, although their pre-test scores were somewhat better than those in, 
say, the Grand Rapids program. The Gary program was dilforent from the others 
in two important respects, however. First, because BEL was responsible for the 
entire curriculum, it could concentrate heavily on reading and math. In tact, almost 
all of the first semester was spent in teaching these subjects. Second, parents of 
Banneker students evidenced much more involvement with their children*s learn- 
ing than did parents elsewhere. Overall, then, the LSCs performed well but did not 
achieve the striking gains-=at least, as measured by standardized testji — expected 
by those educatoi^ who had looked to performance contracting as the golden key to 
compensatory education. 

Why have the achieved gains for 1870-71 been so dififerent from what was 
predicted? There are three possibilities, all of them probably correct in part: 
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1. The LSCs provided instruction that was no more effective than that in 
conventimia] classrooms, 

2. Because the programs were developmental, they could not realize their 
full effectiveness in one year. 

3. The standardized tests did not accu ately measure the results of the per- 
formance contracting programs. 

The simplest explanation is the first. To begin with, both contractors and LEA 
officials set overly ambitious goals* it is no small task, for example, to inspire 
a two-grade»level gain in a single year among students whose previous rate of growth 
was about half a grade leveL But at any rate, the novelty of the systems lay typically 
in the organization of the use of time and materials; the materials and techniques 
were usually well known. Thus, one could argue that the cognitive test scores merely 
reflect that there was little difference between conventional educational programs 
and performance contracting programs. 

This explanation may be part of the story but we think it is only a part. Our 
observations lead us to take seriously the other two alternali ive explanations listed 
above. 

One is that 1970-71 was the developmental year for most of these systems, and 
many problems had to be overcome. It is conceivable that future p'orformance com 
tracting programs will show more substantial gains. We observed many develop- 
mental problems. For example, some of the programs w^ere not well matched to the 
students; some ran into logistic difficulties with facilities and personnel; most 
slighted in-service training; often materials, diagnostic instruments, or the like, 
were excessively late in reaching the teachers" hands; some programs were afflicted 
with high turnover of students, absences, scheduling problems, and dropouts; many 
unmotivated students remained unmotivated; some students did not function well 
in an independent learning situation; some teachers were inept; some LSCs had 
more difficulty than they had anticipated in keeping their programs on the pre- 
scribed track; and some programs were poorly designed. 

Not all these problems were solved in 1970-71; some will likely never be solved. 
A significant number, however, were addressed and it is conceivable that cognitive 
gains may ba higher when teachers and administrators can turn their full attention 
to operational rather than developmental tasks. 

A third explanation of 1970-71 achievement scores is more basic. ^ Some com- 

’ This discussion has profited from conferences with Dr. Richard Hanch of Educational Testing 
Service. 
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mentators question whether standardissed achievement tests can accurately gauge 
the impact of performance contracting (or any other special program) on an in- 
dividual student’s cognitive growth. 

There has been a growing emphasis on the use of student scores on standardized 
achievement tests as a measure of the quality of education. Monies granted under 
Title I of the Elementary and Secondary Education Act carry with them the requira- 
ment for evaluation of the effectiveness with which they have been used. State 
governments have followed suit. Frequently, legislation appropriating special funds 
for education will specif y that project successes be evaluated on the basis of student 
scores on standardized tests. 

This trend is a refreshing change from the traditional evaluation ^ which often 
relied exclusively on reporting whether or not the participants 'Tiked ’ the project. 
The use of standardized teste for evaluation also place,? emphasis on the ultimate 
consumer of education, the student himself 

Despite the attractive features of using standardized test scores as a measure 
of educational output, a number of critics fear they may be misused. In particular, 
the use of achievement test scores as a basis for contractor payment has raised a 
number of objections. 

The basic problem is that such tests were not designed to measure the effects 
of short-run instruction, but to serve as predictive tools.® Given a student’s score on 
a standardized test, one can predict with some confidence what his future perform- 
ance will be in pertinent academic fields. The teste were not designed to distinguish 
the contribution of "good teaching” to success from that of other influences, such as 
the students’ home environment. 

One serious aspect of this problem is that the sorts of behavior a program seeks 
to affect are likely to differ in number and possibly in kind from the behaviors tested 
by the usual instruments. For this reason, standardized teste may not adequately 
measure the results of short-run instructional programs. This is particularly likely 
if a program concentrates on a relatively narrow set of skills (such as word attack) 
that may provide the basis for student mastery of more general skills (such as 
reading comprehension). In such an instance, it is unrealistic to expect standardized 
tests to measure the effectiveness of instruction. One may take issue with the pro* 
gram design, however, if it is this narrow; but even if the program is not narrowly 
restricted, it is still likely that no standardized test will exactly match its content 

® Robert E. Stake, "Tea ting Hazards in Performance Contracting^* Phi Delta Kappaiu J‘ine 1971, pp, 
583-589, 
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and emphasis,^ Stake has suggested that one way around this difficulty is to draw 
selectively from items on a standardized test in order to match the program content, 
but this procedure sacrifices the very feature of standardized tests that makes them 
so useful — the existence of norms. 

Even apart from this basic problem that the measurement tool in current use 
is not well adapted to the task of evaluating the contribution of various instruction 
inputs to student-learning outcomes, standardized achievement test scores as a 
measure of program success have been attacked on other grounds. One problem that 
has received much academic attention is unreliability and bias in computations of 
achievement change.^® The usual discussion dwells on the possibility that change^ 
score computations might indicate more learning than actually took place; we be= 
iieve they might also indicate less, 

The disadvantages of using sta ndardized norm-referenced tests to measure the 
success of performance contracting programs have inspired attempts to construct 
new tests specific to the programs they will measure. These tests are often referred 
to as "mastery^’ or ^'criterion-referenced” tests because student performance is 
judged on the basis of criteria in performing a specific task in relation to a defined 
population. Two of the programs in the study (Norfolk and Texarkana, 1970-71) 
incorporated this feature, but the word was easier than the deed. No one had 
anticipated the tremendous effort that would be required to construct and adminis- 
ter the tests. Typically, questions arrived later than the contract had specified, and 
test administrators had to resort to sampling of both items and students to keep 
resources required for testing within manageable bounds. The other difficulty was 
the questionable reliability of some of the test items, but this is a common problem. 

In spite of these problems, the Norfolk evaluator was able to perform four 
separate administrations of criterion-referenced tests. Since in two of them at least 
90 percent of the students met the criteria, - * the Norfolk program appears to have 

® Ibid, 

Ibid.; L, J. Cronbach and L. Furby, "How We Should Measure "Change’ or Should We?" Psychologi- 
cal Bulletin^ Vol, 74, No. 1, 1970, pp. 68-80; Quinn McNemar, "On Growth Measurement," Educational 
and Psychological MBasuremcnti Vol. 18, No. 1, 1968, pp. 47-66; Frederic M. Lord, "Elementaiy Models 
for Measunng Change,” Problems in Measuring Change, Chester W. Harris (ed.). University of Wisconsin 
Press, 1967, pp. 21-80; Roger T. Lennon, "Accountabiiity and Performance Contracdng," address deliv- 
ered to the American Educational Research Assoc iation, February 5, 1971 (mimeograph); Robert L. 
Thorndike and Eli^beth Hagen, Measurement and Evaluaiion in Psychology and Education, 2d ed., John 
Wiley & Sons, New York, 1961. 

* ‘ See VoL 2 of this study. 




been effective in teaching what it was designed to teach. The difficulty, as indicated 
by the gain scores in Table 4, is that there seems to be a weak relation (at least at 
the elementary level) between the program content and the content of the standard- 
ized tests. In such a ease, the school system must decide which of the results is the 
more signiflcant from the point of view of the needs of the student population. 
We wish to emphasize that the problem of obtaining adequate measurement 
instruments for performance contracting programs is only one of three possible 
explanations for the 1970-71 scores. We have dwelt on testing at greater length 
because we believe that one of the major outcomes of performance contracting in 
1970-71 has been to focus public attention on educational measurement problems. 
These are old problems but the use of test scores to compute contractor payments 
has publicized the need for powerful and accurate evaluation instruments. We also 
believe that the 1970-71 performance contracting experience indicates that this is 
a fertile field for educational research. 



• The performance contracts during 1970-71, the first real year of performance 
contracting, did not produce dramatic gains on standardized achieven’ient tests, 
although in most instances gains were respectable. 

• The performance-contracting movement has focused attention on the prob- 
lems associated with gain scores on standardized achievement tests. 

o Much more work needs to be done on criterion=referenced tests before their 
results can be interpreted meaningfully. 

• Performance contracting has demonstrated that decisionmakers responsible 
for passing judgment on program effectiveness must become more involved in choos- 
ing the instruments that will measure that effectiveness. 



In sum: 
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IV. RESOURCE REQUIREMENTS FOR INSTRUCTIONAL 

PROGRAMS 



; The actual costs of performance contracting programs are discussed in the ease i 

j studies^ Volumes 2 to 6. Such figures are of historical interest but they are not | 

I particularly helpful in answering two questions often asked about performance i 

[ contracting programs. One is, how do the costs of the different programs compare? | 

The other is, how do costs of performance contracting programs compare with costs % 



of other types of instructional programs? | 

To address these questions, a Comparable Replication Cc^t model has been ^ 

constructed/^ This model uses the resources that would be required for an in-house | 

replication of a given learning system. Second, the model holds factor-costs^ — cost per | 

unit of resourees — constant across all programs* that is, the same unit prices are t 

used for each resource in all programs. For example, in the illustrations to be 
presented it is assumed that eveiy elementary teacher is paid a salary of $12,000, 



and this figure is used in all programs. Actual salaries and other local prices of 
course differed widely among programs. Also, the actual programs required re= 
sources that would not be needed for an in=house replication. These considerations 
are important for determining what any given program might cost in any specific 
district, but are not useful for making general comparisons among programs. 

By S. A. Haggart. This section if derived from her r^ource analysis of performance contracting 
programs. 
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To Compare programs w© have computed the comparable replication cost for all 
programs except Gary, which encompassed an entire school. Table 5 gives the esti- 
mates, assuming: 

• The same prices are paid for factors in each district. 

• Classroom space is already available. 

• Program direction is available from the central administration. 

• Curriculum and materials have been developed. 

Table 6 summarizes the resources involved, and Table 7 presents the data used 
to estimate the comparable replication costs. It should be noted that the costs are 
based on modules of a minimum group of students. That is, it is assumed that certain 
facilities are required to accommodate a certain number of students per instruc- 
tional area per day. For example, in the Dorsett Texarkana program the module is 
120 students, or 20 students per classroom for 6 hours per day. If the Dorsett 
program were to handle 150 students, for example, two units would be needed (not 
1.25). If it were applied to 60 students, one unit (not OJ) would be needed. In either 
case there %vould be underutilization. 

The Comparable Replication Costs of the programs varied widely.'^ The range 
was from a low of $103 per student per subject in the CMES program in Grand 
Rapids to a high of $187 in the WLC program in Gilroy. These cost figures, not 
unexpectedly, reflect the Intensity with which the difFerent programs used teachers 
and aides. 

The comparison between performance contracting programs and other types of 
programs is instructive. Tables 5 and 6 show two typical programs, one a remedial 
reading program in Norfolk and the other a regular reading and mathematics 
program in Grand Rapids. As should be expected, the cost per student per subject 
in the conventional Grand Rapids program is quite a bit less than that of the 
performance contracting programs- — about half the Comparable Replication Cost of 
most programs. The Norfolk remedial reading program is much higher in cost * 
$248 compared with $187 for the most expensive performance contracting program. 
The Norfolk approach to compensatory education is very labor-intensive; other 
programs might use fewer teachers. Nonetheless, our observation is that, in general 

Ket-p in mind that Comparable Replication Costs do not include many developmental and adminis- 
trative posts actually Incurred in performance contracting programs, and constant Tactor prices are 
assumed in all districts. They are not the actual costs any ipeciflc LEA or LSC would incur to implement 
a program. 
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performance contracting prograrns compare favorably on a cost-per-studGnt basis 
with the usual remedial programs. 

Cost'per-student is a misleading ratio* of course. One really needs some cost- 
effectiveness measure that takes outputs as well as inputs into account. Perform- 
ance contracting programs should not really be compared with usual classrooms* 
because more effort must be expended to educate students who have had fewer 
academic advantages than their peers. But performance contracting programs com= 
pare favorably with the usual Title I programs, which have had disappointing 
showings so tar. 

One economic feature of performance contracting is brought out in the compari- 
son of the resources in Tables 6 and 7, Compared with a regular program, perform* 
ance contracting programs tend to spend less on certified teachers and more on 
paraprofessionals, materials, and equipment. In economic terms, there is a substitu= 
tion of materials, capital, and less-trained labor for highly trained labor. 



• Comparable replication costs of performance contracting programs vary 
widely. In the Rand sample, the variation was from $103/student/subject to 
$187 /student/ subject. 

. Performance contracting programs cost more than conventional instruction. 
This is to be expected, since their purpose is to make up for the educational disadvan- 
tages of target students. 

• Performance contracting programs cost about the same as (or less than) 
typical remedial programs funded under Title I. This is because the performance 
contracting programs substitute aides, materials, and equipment for highly trained 
and highly paid special teachers, 

• The important consideration, however, is not C 0 Bt-per=student but cost=effec- 
tiveness. Unless cognitive gains in future programs are higher than those to date, 
the extra cost of performance contracting programs will have to be justified on the 
basis of ancillary benefits such as curriculum development potentials. 



In short: 
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EVALUATION 



Most performance contracting prograrnSj including most in the Rand sample, 
have had difficulties with evaluation. Four major points stanvl out: 

• Performance contracting fostered a healthy emphasis on the student and his 
learning as a measure of the success of programs. 

• The requirement for maintaining the integrity of the validation of scores on 
achievement tests sometimes made it difficult to use evaluation data for program 
improvement. 

• Evaluation de^dgns were often haphazard or nonexistent. 

• Data needed for thorough evaluations were usually inaccessible or unavaila= 

ble. 



For the purpose of this discussion, "evaluation/’ is defined here to mean the 
determination of whether a program’s objectives are being or have been achieved 
and determining the reasons for discrepancies between accomplishments and objec- 
tives/^ Validation is a necessary part of any evaluation that uses quantitative 



Stufflebeani points out that educators usually define evaluation . . as the science of determining 
the extent to which objectives have been achieved.” D. L. Stufflabeam, "Evaluation as Enlightenment for 
Decision Making,” in W, B, Beatty (add, Improving Educational Assessment and an Inventory of Measures 
of Affective Behavior, National Educational Association, Washington, D,G., 1969, p. 47. However, evalua- 
tions that seem to have had an effect on decisionmaking about programs or the nature of the programs 
themselves have usually gone beyond mare computation of the discrepancies between goals and attaiiv 
ments and have tried to analyze the wauses of the variance. 



measures, such as achievement gains. We define "validation^' as the measurement 
and certification of the quantitative magnitude of variables. In the performance 
contracting context^ for example, achievement gains must be validated so that 
contractors can receive the monies owed to them. Evaluation in the sense of deter- 
mining whether the objectives of the program were met is a related but broader task. 

Because gain scores have been used to determine contractor payment, there has 
been a great emphasis on validating student scores on standardized achievement 
tests, or in some instances, scores on criterion^referenced tests. Evaluations have 
generally focused on such validation. Some evaluations have assessed noncognitive 
program outcomes such as student attitudes. Even so, as will be shortly discussed, 
this phase of performance contracting evaluations leaves much to be desired. In 
many programs, other relevant evaluation needs have been slighted. 

The major gap has been provision for ongoing evaluation of educational pro- 
cesses to provide relevant managerial information. In some cases this need has not 
been perceived or has been regarded as falling outside the evaluation function. In 
one city in the Rand sample, the efforts to maintain test security and to assure that 
the evaluator remained rigorously objective meant that data that ware needed 
during the program were not gathered or made available to the LEA. 

Evaluations need not and should not be restricted to achievement-score valida- 
tion. The 1970-71 Texarkana program provides a good example of an evaluation that 
served all the needs for information for validation, process evaluation, and decision- 
making, The evaluation was planned in detail before the contract was let, and the 
evaluator provided program administrators with information concerning program 
management, the instructional process, and student achievement throughout the 
development of the program. Probably the bitter experiences of the preceding year 
that amply demonstrated that mere validation is insufficient, coupled with the close 
proximity of a competent evaluator, led to this move. 

In sum, the evaluation of a developmental program should provide two kinds 
of information: (1) information that contributes to the improvement of the program 
as it develops, and (2) information that helps decisionmakers determine whether and 
how the program should be expanded in subsequent years, Such an evaluation must 
be carefully planned to assure that desired data will not be lost as the program 
progresses. Planning should involve not only the evaluator and l^EA program direc- 
tor, but also building principals, those who will be responsible for making decisions 

Rarely are programs terminated, even if the evaluation is unfavorable. See R. A. Levine and A.P. 
Williams, Muking Evaluation Effective: A Guide, The Rand Gorporation, R-788'HEW/CMU, May 1971. 
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about program continuation* and others whtjse support for the program is needed 
for its success and future operation. Plans should explicitly provide for feedback to 
persons influential in program operation as the program develops. 

It may be argued that if an evaluator is this closely associated with the develop- 
ment of a program* his objectivity will be questioned. This did not seem to be the 
case in Texarkana, however* where his objectivity appears to have been accepted. 
If there is such a concern* the use of an auditor in the program may help assure 
accuracy and honesty. 

Turning to the second point* evaluation planning, the Rand sample runs the 
gamut fioin elaborate designs carefully coordinated with the other parte of the 
program (Texarkana, 1970-71) to sketchy evaluations with essentially no plans at 
all. In Gilroy, the formal data-gaihering activities were limited to administering two 
standardized tests: the SAT as the uasis for payment, and the MAT to maintain 
continuity with their previous Title I testing. Other evaluative data were gathered, 
however* because of the participation of Rand and of district employees working for 
degrees. The evaluation of the WLC program in Grand Rapids followed a similar 
course. 

A somewhat more extensive effort was put forth to evaluate the CMES progreLm 
in Grand Rapids, but again the evaluation consisted primarily of comparing pre- and 
post-test scores. An outside auditor verified testing procedures. An assistant princi- 
pal, as part of a graduate course, designed an attitude questionnaire that was 
administered to about 50 pupils several times during the program, but we do not 
know whether these data were used by school officials. 

The Alpha evaluator, Battelle Memorial Institute of Columbus, Ohio* was under 
contract to OEO. Battelle was to provide the district with achievement scores on 
pretests, interim tests, and post-tests. They conducted surveys of parent and student 
attitudes, but were not obliged to turn these data over to the district. 

In Gary* the evaluator conducted the achievement testing and analyzed the 
results. He also prepared some questionnaires on student and parent attitudes. 
Process and managerial data* when they were obtained, had to be generated outside 
the evaluation process on an ad hoc basis. 

The short shrift given most aspects of evaluation (particularly planning) in the 
programs we observed meant that most programs suffered from one or more of the 
following: tests were selected with little regard for their relation to the program; 
'^control*’ groups (if any) were chosen on the basis of criteria diflferent from those by 
which tlie treatment groups were selected; building principals had insufficient no- 
tice for scheduling students and rooms to accommodate test administrations; infor- 
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mation concerning outsiders* interest in and involvement with the program was 
gathered sporadically (if at all); records on disciplinary^ actions and other attitude 
indicators were not kept in usable form; data on IQ or district reading scores were 
collected without regard to the type of tests used; the evaluator did not observe 
actual classroom instruction; no evaluative feedback was provided during the pro= 
gram. If information on the attitudes toward the program of studentSj teachers, and 
others was gathered at all, it was by means of traditional questionnaires, which tend 
to suffer from the usual problems of biased sampling, unreliability, and invalidity. 
In short, only one program had the nearly constant watchdog mechanism needed 
to assess what was actually happening in the program. In no instances were trained 
observers used to monitor classroom activities in order to determine what, in fact, 
went on during the actual instruction* 

The evaluators were hampered by the difficulties of gathering data in almost all 
school districts — difficulties we encountered during our own efforts. Data on stu- 
dents that are funadmental to almost any evaluation— IQ, scores on achievement 
tests, records of dijeiplinary actions, records of* attendance at school or in specific 
classes, history of enrollments at schools in the district, date of birth, notes on 
membership in a minority group, indicators of socioeconomic status such as the 
occupation of the head of the household, identification of sex, and the like— all are 
scattered throughout the system in cumulative folder^, various central files, teach= 
ers* record books, and so on. Retrieving these data requires so much time and effort 
that the task often was not even undertaken. 

Borne of the contractors attempted to gather other data peculiar to their pro- 
grams. Elaborate forms were prepared on which classroom teachers or project direc- 
tors were to record information concerning the time spent in use of materials and 
equipment, visits to learning centers, reasons for student dropout, or other informa- 
tion. In one instance, teachers were required to enter attendance at the learning 
center on four different forms. Duties of this nature were burdensonie to busy 
teachers and administrators, and were often performed sporadically or given over 
to aides. The latter type of afrangement appeared to be the most economical and the 
most likely to result in accurate records. 

There have been evaluation accomplishments as well as problems. Wc have 
dwelt on the problems because successful performance contracting programs re- 
quire thorough evaluations. Evaluation can and should go beyond validation and 
provide the information needed for management decisionmaking, particularly deci- 
sions with respect to turnkeying new systems. Such evaluations are feasible, as the 
more outstanding evaluations to date have shown. Decision-relevant evaluations are 
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a challenge and require significant expenditures. Considering the costs of perform- 
ance contracting programs and their potential impacts^ thorough decision-oriented 
evaluations have proven themselves to be well worth the effort and money* even 
though they may well be more expensive relative to program cost than the conven- 
tional 10 percent For example, the Texarkana evaluation probably cost about 1/6 
of the total program cost‘« in 1970-71. 

Including all program costs, not merely the payment to the contractor. 
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VI. PROGRAM MANAGEMENT 



Performance contracting appears to be a useful R&D tool, particularly for 
smaller school district without an abundance of resources for program develop- 
ment. Porfbrmanc© contracting is a useful way for a school district to explore new 
programs and to implement those that appear promisiag- People who are not a 
permanent part of the school system seem to be freer to implement radical changes 
in the classroom than are regular school personneL Gary is the most obvious exam- 
ple of thiSj but the hardware-oriented systems in Texarkana were also significant 
departures from regular practice, as was the hi tensive CMES program. 

One point that stands out in each of the five case studies is thaL. the successes, 
failures, and problems associated with each program were intertwined with the 
personal characteristics of the people in charge and the intensity of their commit- 
ment to the program. True, this can be said of nearly all human endeavors, but it 
has an important implication for performance contracting. Any program that seeks 
to change the status quo has a hard task. A strong central figure who is personally 
committed to the success of the program can tip tl balance between success and 
failure. Fortunately, committed people who were strong managers pushed the pro- 
grams in all of our case studies. Some were supermtendents or assistant superin- 
tendents of schools, some were program directors for the school system, and in at 
least one case the head of the Model Cities program was the spur. We have observed 
other programs languishing for lack of such a spur. If we were asked to name the 
single thing that an LEA or LSC could do to make a peiTormance contract an 




efTeetive change agent we would air-wen make sure that someone in tl r LV]A*s 
administration with ability, position, and respect adopts the pirogram as his personal 
project. 

In most districts, the need for extensive program changes had not been an- 
ticipated nor had the contract provisions and program administration been designed 
to accommodate such changes. In some instances, contracts had to be revised or 
amended part way through the program in order to adjust to unavoidable ciiunges. 
These revipons always required considerable effort from most parties that had 
contractual arrangements associated with the program. Consequently, the contracts 
as they were written actually hampered the deveionment effort. 

The developmental nature of performance contracting programs implies that 
both LEAs and LSCs should build flexibility into programs and contracts. It also 
means that programs that last a single year or less labor under a great handicap, 
because most of the time must be spent cn tailoring the system to the school district 
find its students. Multiyear projects, such as the four-year Gary program and the 
five-year Texarkana program, have considerable merit because they permit pro= 
gram redesign on the basis of the feedback from the first year or two. 

Performance contracting programs have placed new demands upon school dis- 
tricts for management and coordination. Organizing a program, dealing with re- 
quests for proposals (RFPs), contract negot atioris, fees, and other such features of 
many programs have required considerable l Tort from LEA officials, and often more 
time than had been anticipated. 

The same generalization applies to the operational stage of the programs. Trou- 
bleshooting was required by both LEAs and LSCs. There was a great deal of effort 
by aii involved in these programs to make them work. In part, this is a feature of 
conscientious program administration; but beyond that the special accountability 
feature of performance contracting is a built-in spur for administrators to examine 
continuously, assess the progress of the progi-am, and make changes. 

The difficulties of managing the developmental programs were heightened, in 
several eases, by a proliferation of contracts that fraiiqnented responsibility and 
authority among several parties. The more elaborate the ' ontractual arrangementa 
were, the harder it was to maintain a clear picture of who was responsible for what, 
who was doing what, who had talked to whom, and so on. In other words, the familiar 
problems assoeiated with an overextended span of control arose, and the school 
sptem's project director had to divci t energies away from his primary responsibili- 
ties merely to provide liaison among the organizations involved. The solution to such 
situaticns -to develop the needed skills within the school S3^tem— will take time. 




Legal questions about the conformity of performance contracts with state edu- 
cation codes and procurement regulations arose during 1970*71, but for the most 
part were letl unresolved on the grounds that these were experimental programs, 
The same generalization applies to conflicts between performance contracts and 
ji.EA-union contracts or established teacher employment practices. In future years 
challengers are less likely to hold their Are; courts, state departments of education^ 
and legislatures will no doubt be called upon to settle disputes. 

An analysis of the legal and union issues that arose in Gary^ and our studies 
elsewhere, indicate that most legal difflculties can be resolved with the expenditure 
of some effort, Performance contracts may encounter two matters of principle, 
however, that can create irrembdufot;^ difficulties. The first is the issue of the cogni= 
zance and control of instruction. Uc ’ the LEA firmly establishes and exercises its 

ultimate authority over a program, it is going to encounter legal diflicuities that 
cannot be resolved short of restructuring the program in a way that conclr.sively 
nails down its authority. 

The other major matter of principle involves union opposition to merit pay. 
Unions feel that differential staffing smacks of merit pay, although they have ac- 
cepted it in a number of cities. Any special payments to teachers, however, are going 
to nm into serious opposition, as the Grand Rapids eKperience with the Alpha 
program showed. 

Teacher groups with eollective bargaining agreements are likely to demand a 
voice in negotiating and structuring contracts on the basis that the performance 
contracts change conditions of work. (In 1970=71 their demands were not met in the 
cities in the Rand sample.) Also, programs that drastically change class sizes are 
likely to produce difficulties. Most such problems can be resolved but special incen- 
tives to teachers are likely to produce stalemate.” 

Another lesson from the 1970=71 experience is the need to involve the teachers 
in design and planning for all phases of the program. Most contractors used local 
school system personnel to teach (all the programs in the Hand sample fall in this 
category).*® The teachers, how'ever, have had relatively little to say about the system 
at the start of the programs. In some cases the reason seems to have been lack of 

’’ Union:i stiern reluctantly willing to accept the idea that teachers in performance con tf acting 
programs may have a greater income than that of the usual teacher if the difference is due to (•vertime 
as speciiled in the contract. 

In Texarkana the teachers were not on the LEA's payroll but they were eligible for credentials and 
were hired from a list that Texarkana maintained of qualified applicants for employment. Some were 
not yet credentialed. 
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time; in others it appears the LSC was proud of its system and believed that it was 
teacher-proof As it turned out, the LSC increasingly involved the teachers in 
planning and administration as the year went on, to the benefit of the programs. 

In most programs, little effort was made to inform parents about the goals and 
techniques of the programs or to involve them in progiam planning, monitoring, or 
evaluation. Often parents were unaware that their children were in a special pro- 
gram, sometimes pareid^- were confused by such features as nongraded report cards 
or were hostile to classroom '^permissiveness/' It is possible that the superficiality 
of rnost gestures toward parent involvement contributed to generally disappointing 
achievement scores. 

In sum: 

• Performance contracting is proving to be a useful research and development 
tool. People who are not a permanent pai t pf the school system seem to be freer to 
implement radical changes in the clnssroom than are regular school personnel, 

• A respected and influential "sponsor” within the school district is a great 
help in overcoming inherent inctions and impediments to change. To assure that 
changes are permanent and that ':hey expand beyond isolated programs will require 
continued high levels of sustained LEA effort. 

• Flexibility is essential in program organisation and management, since con- 
siderable program development will take place. Multiyear programs have advan* 
tages over single-year programs for this reason, 

• Performance contracting programs impose special tasks of managemen t and 
coordination not only on contractors but on the schools' administrative personnel. 
The complexity of some programs has exacerbated the.se problems. 

• administrators must be prepared to face legal and labor disputes. 
Most of these can be resolved but there are two potential areas for serious conflict. 
One is the requirement for public control of a//school programs. The other is teacher 
opposition to merit pay. Contractors must meet greater demands from school dis- 
tricts for responsibility for the goods and services they sell. The performance con- 
tract encourages active contractor participation in the program modifications that 
are generally required to adapt to the particular needs of a district. 

• It seems essential that local teachers be involved in program design and 
administration, 

• Little effort was made in most programs to inform parents of the programs 
or to involve them. Many parents were confused by or hostile to some aspects of some 
programs. 




VII. RETURNS TO THE CONTRACTORS 



Superficially, the primary return to the contractor in a performance contract 
is his payment for student achievement. Actually, it begins to appear that contrac- 
tors are realizing much more significant gains in the form of follow-on contracts for 
materials and servicei for the coming school year, only some of which will be 
perfbrinance contracts. Norfolk has already bought LRA/UT^T matcxmls and equip- 
ment for eight more reading centers, with no pertormance contract involved; Texar- 
kana will use EDL materials and equipment in a modification of the turnkey pro= 
gram, now referred to as individualized instruction. Even though WLC has phased 
out its School Operations Division, which ran last year's programs in Gilroy and 
Grand Rapids, the WLC centers under the 1970-71 contract in Grand Rapids will be 
operated under other auspices and three more centers will be in operation. Two were 
opened halfway through the 1970-71 school year at the teachers' instigation. Learn- 
ing Unlimited will manage the centers in Grand Rapids at a flat rate per student. 
If Gilroy has a center it will be run by teachers. 

Three of the programs will continue as performance contracts v/ith the same 
contractors: BRL in Gary, because it was a four-year program, CMES in Grand 
Rapids, and Alpha in Grand Rapids. Alpha will expand its operations in several 
areas. Texarkana, apparently, will again go through the process of issuing an RFP 
and selecting a contractor for next year's performance-contracted program. 

It is not likely that any of the contractors involved in the programs in the Rand 
sample enjoyed a high rate of return on their investments. This conclusion is a 



speculation, of course, since information on LSC costs is proprietary, but achieve- 
ment gains were not sufficient to generate large bonuses and sizable development 
and managerial expenses v;ere encountered. Rand's field study supports Sigefs 
conclusion that, *'The rub is there's little profit, if any, in running programs with 
the amounts schools now spend.” 

One of the leading contractors, Westinghouse Learning Corporation, with two 
programs in the Rand sample, has discontinued its participation in performance 
contracting. Other performance contractors have indicated to us that they are going 
to be more particular about the financial arrangements in future programs. 

Performance contracting has not been an immediate bonanza, but it has been 
very rewarding for contractors in another way, having helped a number of firms in 
the education industry to break into nev;- nm»*Wets heretofore largely dominated by 
textbook publishers. The key to overcoming the barriers to entry was the LSCs' 
willingness to offer sO'Called "guarantees”— promises of refunds if achievement 
gains were not realized. Many LEAs have wanted more contractor involvement in 
implementing the systems they sell and more assurance of benefits if scarce educa- 
tion doilai s are Spent. The claim heard in 1970=71 that business was "trying to take 
over the classroom”"^ was somewhat off the mark. Disillusioned LEAs have gener^ 
ated much of the pressure for performance contracts. Many a dazzled or desperate 
school district has purchased equipment and rnaterials because they were touted as 
wondrous, only to consign them to the attic, perhaps because they were inefTective, 
perhaps because the teachers did not use them properly. In any event, LEAs have 
wanted more follow-through and warranties for new systems, and at the same time 
many LSCs have been seeking opportunities to demonstrate their wares. 

The mechanism of the performance "guarantee” fitted into both desires. From 
the contractor’s point of view, the public exposure this year has made performance 
contracting ventures worthwhile (at least for some of them). It is a.lso obvious that 
established firms are going to be less anxious than newcomers to use this technique. 

In brief: 

• Performance contracting does not seem to have generated large profits so far. 

• Performance contracting has generated some follow-on programs, only some 
of which tie fees to student achievement. 



** E. Sigel. Accountabiliiy and the Controversial Role of the Performance Contractors. Knowledge 
Industry Publications, Inc., White Plains, New York, 1971, p. iii. 

R. D. Bhaerman, "Accountability: The Great Day of Judgment," Educational Technology, Vol. 2, 
No, 1. January 1971, pp, 62*63; "Hucksters in the Schools; The Performangi; Contracting Phenomenon." 
American Teacher, Vol. S4, No. 1, September 1970, pp. 9=11. 
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. Established contractors tend to prefer other arrangements to performance 
contracting, such as consultantships. Performance contractors will seek to convert 

performance coDiiracts to other types of programs. ... 

. Performance contracts have enabled a number of firms to break into new 
markets and to receive publicity for their goods and services. Therefore, they will 
coi tinue to be let. 




s.- 
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VIII. MAJOR ADVANTAGES AND DISADVANTAGES OF 
PERFORMANCE CONTRACTING 



To summarize the foregoing discussions, we have consider the major advantages 
and disadvantages of performance contracting in education that can be inferred 
from our field work. The major advantages appear to be three: 

• Performance contracting facilitates the introduction of radical change m 
education. Outsiders appear to be freer to implement new methods and materials 
than are regular school employees. In addition, small school systems cannot afford 
the capital investment required for initial research and development of new instruc- 
tional approaches, which LSCs may be able to recoup by future sales. 

• Performance contractin g places increased emphasis on accountability for stu- 
dent learning. School administrators are faced with the requirement to define objec= 
tives for education, and at the least to understand the relationships among objec- 
tives, instruction, and the tools used to measure the effects of instruction. They have 
become more aware, also, of the cost of programs, and of the pressing need for the 
ability to gauge the various outcomes of a program against its cos^— that is, to 
measure cost-effectiveness — which would be of great value for allocating resources 
and for estimating what levels of achievement future programs can reasonably be 
expected to attain. Contractors, likewise, have become involved in the use of the 
products they have so fervently promoted. They have had to cope with the logistic 
and managerial problems of adapting their products to actual school situations and 
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the realities of test selection and administration. They, too, have been made more 
accountabl&=more aware of the relationship between prograni cost and outcomes. 

• Performance contracting has brought new LSCs into the educationQl field. 
This point has been made previously and need not be expanded here. 

Three major disadvantages also seem evident: 

• Some performance contracting programs have been so complex that manage- 
ment has been set. rely hampered and costs have been unnecessarily high. Complex 
contracts have been engendered by an attempt to contract for skills thought to be 
lacking wdthin the district. Management support groups, independent evaluators, 
and educational auditors have all been used. The advisable solution to lack of skills 
seems to be to attempt to develop them within the district, rather than hire them 
ad hoc and pieeejmeal. 

• Performance contracting programs will probably continue to be narrowly 
focused because of difflculties of de finning objectives in subject areas other than those 
involving simple skills or^ in some cases, difficulties in measuring the attainment of 
objectives. This point has also been sufficiently explored above. 

• Performance contracting has exacerbated old problems to the point where they 
almost seem to be fiew ones. There have always been problems with managing 
program.^, but the inescapable need to provide a given number of instructional hours 
to a given number of students has highlighted start-up difficulties. In addition, 
writing RFPs, negotiating and renegotiating contracts, and resolving diiTeronces 
concerning lines of authority and responsibility, all become more crucial in a per- 
formance contract. 

Special legal difficulties have arisen and are likely to become more severe in the 
future. Owing to the pressure of deadlines for getting programs adopted, organized, 
and launched, legal shortcuts were sometimes taken, education codes have been 
challenged, state-adopted textbooks discarded, and local laws violated. Sir ce these 
illegalities can be used by opponents of performance contracting, LEAs and LSCs 
will continue to need legal help to forestall disruption of the programs. 

Since 'liiost progra is attempt to individualize instru tion, they must lower costs 
by Bubstitiiting materials, equipment, and aides for highly trained teachers. Doing 
so changes the role of teachers, who often become discontented or enter into outright 
opposition. 

Finally, performance contracting has spotlighted the inadequacy of gain scores 
on standaidized achievement tests as measures of’ the effectiveness of instructional 
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programs. More reliable and valid measuring instruments in a wider variety of 
instructional areas must be devised if performance contracting is to become a widely 
used tool in education. 
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Appendix 

PROGRAM AND RESOURCE INFORMATION 



GARY 

Behavioral Research Laboratories 



Charac?tsr*istiss of students 



^nogpam soons 

Instruction 

Number of students . . , 

Class tlme^ 

Class size 

Number of sections ^ , . , , 

Facitt-ttes 

Space. , 

Students/classrooTn/day 

Furnishings 



Staffing 

Certified teachers 



Special teachers 
Aides ... ....... 

Other ........... 



Equipnent 

Mateidats 



Pra-s&rvioe training 

In-sevvise training - . , . 



Other support 
Inaentivea , . . 



Gradas K-6; black, lover-middle— class 
family; low transiency rate 

All subjects 
850 

Entire school day 
Variable 
20 



Entire school 

Variable 

Conventional 

5 curriculum managers (master teachers); 
20-5 assistant curriculum managers (other 
teachers) 

2 curriculum consultants 
20 

1 program director 

No special equipment; 1 reading labj oper= 
aLcid mornings 

BRL-Sulllvan Project Readj Project Math; 
SoienoBi A Froaess Approaoh (AAAS-Kerox) ; 
Man^ A CourBe Study \ other standard 
Indiana texts 

2 weeks 

4 weeks plus continuing activities of 2 
full-time consultants 

None 

None 
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GILROY 

Wes tin gh ouse Learning Corporation 



•la^^aterist-icQ of students 



Pno0i'u/fi 3<Jope 

Class time , ^ ^ ^ ^ ^ 

Class size . 

Number of sections 

Utilization 

Number of students 



Grades 2-4; Title I; low socloeGonomiG 
status 



1.25 reading, 1.25 math 
50 students per class 
2 

5 hours a day 
103 



FaaiZitiBS 

Space. 2000 sq ft; 1 elassroem, 1 activity area 

Furnishings........... 6 carrels, carpeting, tables 



Staffing 

Special taachars 1 f ull- tiiue^equivalent reading speGialist 

Paraprofessionals , 2 par centar, 1 par activity area 



Equipment 



TeleXi tape recorders, cassette players, 
headset 



Mater*iats . . 

Pi^s^seY'vioe training , , , 

Tn-Bervioe training 

Other support 



Books i games , toys 
4 days 

1 week, total 

Ratnote diagnostic and prescriptive 



incentives 



25 per student—candy j scrip 
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GRAI^ID Fy^PIDS 

Ale'Kander Rnd Hall Schools 
(Same for both schools) 

Alpha Systems 



of .^tudontB Grades 1-3; inner-city black (Alexander); 

largely Latin American (Hall); low incoine 
transiency 20%; loi-?est achievers 



Pr^ogT^am su^cpe 

Instruction. Reading and math 

Number of students 300 (100 eacli grade) 

Class time, 2*1/4 hours 

Class size 150 in 3 rooms 

Number of sections 2 

FclC'L Ztti&e 

space. 

S tudents/classroom/day 
Furnishings . ^ ^ ^ ^ - 

Staffing 

Certif led teachers , , > , 

Special teachers 

Aides ^ 

Other a 

EquipmeKt . No special equipment; fret* room game 

equipment 

j Material e Variety of programmed instructional 

I materials (17% consumable) ; free room 

i materials 

i 

I Pre-service training 2 weeks on curriculum, behavior modif ica- 

i tion, contingency management, diagnosis 

and prescription 

In-service training,.,.,. 4 days during year In lieu of regular 

distri et in-service 

( 

I 

! OtJuir support None 

InaentiveB None 



3 regular clabsi . oins 
100 

From regular program 



2 

0 

. . . 3 

1 full=Ciine on-site director, shared 




^itli West Middle School. 
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GRAND RAPIDS 
West Middle School 
Alpha Systems 



C'na.ra<jt&T'lQ of studGni^ , . . Grades 7”9; inner-city racially mixed; low- 

medium Income; transiency 20%; lowest 
achievers 



p~rogx*ojn eoope 

Ins cruet ion < 

Number of students- 

Class time ^ . 

Class size 

Number of sections - 

Fao% Ivties 

Space 

Students/ classroom/day 

Furnishings* * 

Staffing 

Certified teachers 

Special teachers 

Aides 

ocher. * 

Equipnent 



f4atBTia'lQ 



p-p&^sBn’OicB training 



In^wevvCoe training 



Other support 
Inaentives , . . 



Reading and math 
300 (100 each grade) 
110 minutes a day 
lOQ in 3 rooms 
3 



3 regular classrooms 
100 

From regular program 

3 
0 
5 

1 full-time on-site director , shared 

No special equipment; free room game 
equipment 

Varlecy of programmed Instructional 
materials (73% consumable) ; free room 
materials 

2 weeks on curriculum, behavior modlflca^ 
cion j contingency management , diagnosis 
and prescription 

4 days during year in lieu of regular 
district In-service 

None 

None 



^ith Alexander and Hall Schools. 
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GRAND RAPIDS 
South Middle Sehool 



Combined Hotiv-ition and Educational Systems , Inc - 



Oh-^aatei^istias of Gtzidonts Grades 6-9; transiency 26%; blacky model- 

cities neighborhood ; low income ; lowes t 
achievers according to last spring’s testing; 
special education pupils included; program 
pupils distributed among all homerooms 

Prag^ani scope 

No, of students (mid-December),, 491, reading; 535, math 

Glass time, 45 minutes per day each, reading and math 

Class size* , 35-40 in Single Center (SC) (40 optimum); 

60=65 in Double Center (DC) (optlmutn) 

Number of sections,..,, 14 each (7=perlod day) 



FaedLi^tdes 

Space, 4 centers j 1 DC for reading and math; 1 SC 

for reading and 1 SG for math; each center 
has an instructional and an AMS area; 1 reiu- 
forcement room; total occupies space of 7 
former classrooms (walls ware changed) 

Students/classi*oom/day Number of enrollments/number of classrooms - 

(491 + 535)/7 ^ 147 

Furnishings Table space for carrels; carpeting; air 

conditioning; 1 carrel per student per 
class (approximately 140 tota 1) ; chairs 

Staffing 

Certified teachers...,, 1 per center (i.e,, 1 per SC, 2 per DC) 

Special teachers 0 

Paraprof essionals Full time i 1 per cencer, 1 for reinforcement 

room, 1 substitute 

Other personnel..,.,.......,.,., 1 full=tim© director* 1 full=Clme secretary 



Equipment 

Primary unit Reading: 40 Hoffman reading machines; 

math: 40 tape recorders/ center (80 total) 

Supplementary system Reading: 25 tape recorders /center (50 

total); math: 40 flashcard readers 

(Electronic Futures ) 

Redundant system,.,,,..,., 15 Borg-Warner System 80 for reading and math 

Materiats (10^ consumable) 

Reading,,,*,,,...*..... 2 sets EPL tapes per center; 2 sets Hoffman 

materials (levels B to G) per center; workbooks 

Hath, Hath mini-system (tapes) ; workbooks 

Reading and math,,,,,,., 2 sacs Bdrg=Warner materials (levels 1^8) per 

reading and math center (I.e.* 4 complete sets) 
1 notebook per student for compiling materials 

training 1 week on In-depth training, 1 week going 

through materials 



in^servicB tj*aining 

Other BUpport * , , 

Inomntiv&a 



About 2 hour a weak 



Nojjpii-; instractlonal program self-contained 
None 



GR^^ND RAPIDS 
Franklin School 



Westinghouse 


Learning Corporation 


Ckavaat&rvstiaB of studenta 


. Grades 1^6; Inner—city black; law Income; 
transiency 30%; lowest achievers for first 
5 months 5 then entire school (excluding 
most special education students) 


Progrcan scope 

Instruction 

Number of students 

Class time . 


Reading and math 

Inltiaily 100, later 150 (as of February) 
Initially 2-1/4 hr^ later reduced to 75 min 
for grades 1-4 


Glass sise^ 

Number of sections. 


45-55 (maximum at 60) 

3 (1 each for grades 1 and 4, 2 and 5, 
3 and 6) 


Fad t%t%08 

Space 

Students/classropm/day 

Furnishings 


2 regular classrooms 
75 

30 carrels and chairs, with electric out- 
lets at each carrel; 7 tables, 21 chairs; 

3 bookslielf-cabinets ; carpeting 


Staffing 

Certified teachers ^ * 

Special teachers. 

Paraprof essionals 

Other personnel 


1 (no planning time required) 
None 

2 , on 6^hour day 

On-site director and secretary 


Equipment 

Telex 

Cassette tape recorders. 


1 

30 


Materia Is 


BRL modern math texts; large variety of 
other n^terlals 


Prs^serviae training i 


5 days for all teachers of school 


In^Bcrvice training 


d morning meetings for entire staff 


Other support 


None 


Incentives 


None 
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NORFOLK 



Learning Research dissociates 



I 

I 

i 



Ctiai'^aoteria ties of stud^>ft3 , . ^ ^ . 
Progrwn scope 

Nmnbar of students 

Class time 

Class size . . 

Number of sections 

Paa-Hities 

Space ^ 

Students /classroom/day * * . 

Utilization - . 

Furnishings 



Staffing 

Certified teachers 

Special teachere 

Paraprofessionals 

Other personnel 



Equipirient 



Mateviats 

Pve-‘Sev'0%o& travning 

In-sevvioe tva%n%ng 

Othev support 

Incentives 



Title I 



250 

50 minutes 

25 

5 



Regular classrooms 

125 

87% 

Air conditioning* plaaflanc environment; 
small, modern (partitions* file cabinets, 
storage cabinets* etc., loose tables, 
chairs) 



1 per classroom 
0 

1 per classroom 
1 pregram director 

6 cassette players ($25) , 6 tape recorders 
($150) * earphones ($50) 

Sound filmstrip sets* cassettes; workbooks 
and miscellaneous supplies, books, kits 

1 week 

3 days 

Evaluation — $10 per child 

300 paperback books given as awards 



I 
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NORFOLK 



Ramedial Program 




Ckax^aaterist^os of Btud&ntQ , , ^ ^ ^ , Title I 



Progrcmi soope 

Number of students 



Glass time. 

Class size 

Number of sections 



IQOO (14 elementary centers, 60 students 
per center; 3 junior high centers, 50 
Students per center 
50 minutes 
10 
6 



Faaitities 

Space - 

Students /classroom/day . ........ 

Utilisation , , , , 

Furuishingf> 



1/2 regular classroom sise 
60 
80% 

File cabinet, loose tables, chairs 



Staffing 

Certified teachers . 

Special teachers 

Paraprofesalonals 

Other personnel 



i per classroom 
0 
0 

1 program director 



Equipment Language Master 

Materials Books, filmstrips , gameu, kits 

Other cupport . . ^ ^ ^ Diagnoatle center: $35,000 (3 dlagnostl 

clans, 1 aide, 1 clerk, consumables) 



Incentives None 
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TEXAR^NA 

Dorset t Educational Sy8 terns , Inc . 



Cha:)^aatQi-^istiG3 of etudoyits Grades 7*12; educationally disadvantaged 

(at least 2 years below grade level); IQ^ 
at least 75 



Fragram scops , 

Number of studanc8^ 350. reading and math 

Class time. 1 period math, 1 period reading 

Class size........ 20 students par classroom area 

Facttities 

Space 4 trailers, each 900 sq ft; 2 classrooms, 

each 1000 sq ft 

Utiligation. 100 piarcent 

Furnishings...,,. Desks, carpet, air conditioning 

Staffing 

Certified teachers. 1 per center 

Special teachers 0 

Paraprof essionals 1 per center 

Other personnel.,..,.. Project manager 

Equipm&nt Dorsett K*86 Teaching Machines 

MatBT-iats Filmstrips, records, programmed teKts 

P2}e-se.pviae tT^aining 8 days per teacher 

In-sevoioc training No formal training 

Inccnti.vBQ ^ 

Students, Green Stamts, transistor; radios, 1 TV, 

some popular records (and player), games, 
pussies, popular magasines, free time 
Teachers Dorsett stock bonus and options 



* 



SO 




o 



TEXARKANA 



Educati-onal Development Laboratories 



Chaipaots-pistias of stud&nts 



Frog^pom soope 

Number of studenrs' * . , ^ 

Class time. . . . 

Class slse^ . ^ . . . 

Faatttti.sB 

Space 

Utilisation 

Furnls'.ilngs 

Staffing 

Certified teachers 

Special teachers 

Faraprof esslonals . . « 

Other staff. ................... 

Equipment 

Materials ... 



FrB^s&r'OiQe training ^ ............. 

In-servioe training 

Other support 

Inaentives 



Grades 7 *12 1 edticationally handicapped 
(at least 2 years below grade level) IQ 
at least 75 



251, reading^ 261, math 
1 period math, 1 period reading 
20 students per classroom area 



4 trailers , each 900 sq f t j 1 elassrooin 
1000 sq ft 

100% (6 hours a day) 

Desks, carrels, carpet, air conditioning 

1 per center 

0 

1 per center 
Project inanagar 

EDL Aud“X , Tach-X, Controlled Readers , 
Flash-X 

Filmstrips, cassettes, cards with magnetic 
scrips 

40 hours per teacher and aide 

No formal training 

None 

None 



^lodal Cities funded 110 6th=graders for the same instructional program, 
giving a total program of 395 students . 
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